Hallucination of Speech Recognition Errors With Sequence to Sequence Learning

Prior work in this domain has focused on modeling errors at the phonetic level, while using a lexicon to convert phones words, usually accompanied by an FST Language model. We present novel end-to-end models directly predict hallucinated ASR word sequence outputs, conditioning input as well corresponding phoneme sequence. This improves prior published results for recall of from in-domain system’s transcription unseen data, out-of-domain transcriptions audio unrelated task, additionally exploring in-between scenario when limited characterization data test system is obtainable. To verify extrinsic validity method, we also use our augment training spoken question classifier, finding that they enable robustness real downstream scarce or even zero task-specific was available train-time.